{
 "metadata": {
  "signature": "sha256:30447a58e88d72e5e590998e7d24bec548893a3e3f447f1d3a897634c97be2cd"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "MetaArray\n",
      "=========\n",
      "\n",
      "!MetaArray is a class that extends ndarray, adding support for per-axis\n",
      "meta data storage. This class is useful for storing data arrays along\n",
      "with units, axis names, column names, axis values, etc. !MetaArray\n",
      "objects can be indexed and sliced arbitrarily using named axes and\n",
      "columns.\n",
      "\n",
      "Download here: [MetaArray.py](files/attachments/MetaArray/MetaArray.py)\n",
      "\n",
      "Example Uses\n",
      "------------\n",
      "\n",
      "Here is an example of the type of data one might store with !MetaArray:\n",
      "\n",
      "![](files/attachments/MetaArray/example.png)\n",
      "\n",
      "Notice that each axis is named and can store different types of meta\n",
      "information: \\* The Signal axis has named columns with different units\n",
      "for each column \\* The Time axis associates a numerical value with each\n",
      "row \\* The Trial axis uses normal integer indexes\n",
      "\n",
      "Data from this array can be accessed many different ways:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "data[0, 1, 1]\n",
      "data[:, \"Voltage 1\", 0]\n",
      "data[\"Trial\":1, \"Signal\":\"Voltage 0\"]\n",
      "data[\"Time\":slice(3,7)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "Features\n",
      "--------\n",
      "\n",
      "`*\u00a0Per\u00a0axis\u00a0meta-information:`\\\n",
      "`\u00a0*\u00a0Named\u00a0axes`\\\n",
      "`\u00a0*\u00a0Numerical\u00a0values\u00a0with\u00a0units\u00a0(e.g.,\u00a0\"Time\"\u00a0axis\u00a0above)`\\\n",
      "`\u00a0*\u00a0Column\u00a0names/units\u00a0(e.g.,\u00a0\"Signal\"\u00a0axis\u00a0above)`\\\n",
      "`*\u00a0Indexing\u00a0by\u00a0name:`\\\n",
      "`\u00a0*\u00a0Index\u00a0each\u00a0axis\u00a0by\u00a0name,\u00a0so\u00a0there\u00a0is\u00a0no\u00a0need\u00a0to\u00a0remember\u00a0order\u00a0of\u00a0axes`\\\n",
      "`\u00a0*\u00a0Within\u00a0an\u00a0axis,\u00a0index\u00a0each\u00a0column\u00a0by\u00a0name,\u00a0so\u00a0there\u00a0is\u00a0no\u00a0need\u00a0to\u00a0remember\u00a0the\u00a0order\u00a0of\u00a0columns`\\\n",
      "`*\u00a0Read/write\u00a0files\u00a0easily`\\\n",
      "`*\u00a0Append,\u00a0extend,\u00a0and\u00a0sort\u00a0convenience\u00a0functions`\n",
      "\n",
      "Documentation\n",
      "-------------\n",
      "\n",
      "### Instantiation\n",
      "\n",
      "`\u00a0Accepted\u00a0Syntaxes:\u00a0`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      " # Constructs MetaArray from a preexisting ndarray and info list\n",
      "MetaArray(ndarray, info)\n",
      "\n",
      " # Constructs MetaArray using empty(shape, dtype=type) and info list\n",
      "MetaArray((shape), dtype=type, info)\n",
      "\n",
      " # Constructs MetaArray from file written using MetaArray.write()\n",
      "MetaArray(file='fileName')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`\u00a0info\u00a0parameter:\u00a0This\u00a0parameter\u00a0specifies\u00a0the\u00a0entire\u00a0set\u00a0of\u00a0meta\u00a0data\u00a0for\u00a0this\u00a0!MetaArray\u00a0and\u00a0must\u00a0follow\u00a0a\u00a0specific\u00a0format.\u00a0First,\u00a0info\u00a0is\u00a0a\u00a0list\u00a0of\u00a0axis\u00a0descriptions:`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "    info=[axis1, axis2, axis3...]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`\u00a0Each\u00a0axis\u00a0description\u00a0is\u00a0a\u00a0dict\u00a0which\u00a0may\u00a0contain:`\\\n",
      "`\u00a0\u00a0\u00a0*\u00a0\"name\":\u00a0the\u00a0name\u00a0of\u00a0the\u00a0axis`\\\n",
      "`\u00a0\u00a0\u00a0*\u00a0\"values\":\u00a0a\u00a0list\u00a0or\u00a01D\u00a0ndarray\u00a0of\u00a0values,\u00a0one\u00a0per\u00a0index\u00a0in\u00a0the\u00a0axis`\\\n",
      "`\u00a0\u00a0\u00a0*\u00a0\"cols\":\u00a0a\u00a0list\u00a0of\u00a0column\u00a0descriptions\u00a0[col1,\u00a0col2,\u00a0col3,...]`\\\n",
      "`\u00a0\u00a0\u00a0*\u00a0\"units\":\u00a0the\u00a0units\u00a0associated\u00a0with\u00a0the\u00a0numbers\u00a0listed\u00a0in\u00a0\"values\"`\\\n",
      "`\u00a0All\u00a0of\u00a0these\u00a0parameters\u00a0are\u00a0optional.\u00a0A\u00a0column\u00a0description,\u00a0likewise,\u00a0is\u00a0a\u00a0dict\u00a0which\u00a0may\u00a0contain:`\\\n",
      "`\u00a0\u00a0\u00a0*\u00a0\"name\":\u00a0the\u00a0name\u00a0of\u00a0the\u00a0column`\\\n",
      "`\u00a0\u00a0\u00a0*\u00a0\"units\":\u00a0the\u00a0units\u00a0for\u00a0all\u00a0values\u00a0under\u00a0this\u00a0column`\\\n",
      "`\u00a0In\u00a0the\u00a0case\u00a0where\u00a0meta\u00a0information\u00a0is\u00a0to\u00a0apply\u00a0to\u00a0the\u00a0entire\u00a0array,\u00a0(for\u00a0example,\u00a0if\u00a0the\u00a0entire\u00a0array\u00a0uses\u00a0the\u00a0same\u00a0units)\u00a0simply\u00a0add\u00a0an\u00a0extra\u00a0axis\u00a0description\u00a0to\u00a0the\u00a0end\u00a0of\u00a0the\u00a0info\u00a0list.\u00a0All\u00a0dicts\u00a0may\u00a0contain\u00a0any\u00a0extra\u00a0information\u00a0you\u00a0want.`\n",
      "\n",
      "`\u00a0For\u00a0example,\u00a0the\u00a0data\u00a0set\u00a0shown\u00a0above\u00a0would\u00a0look\u00a0like:`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "  MetaArray((3, 6, 3), dtype=float, info=[\n",
      "    {\"name\": \"Signal\", \"cols\": [\n",
      "        {\"name\": \"Voltage 0\", \"units\": \"V\"},\n",
      "        {\"name\": \"Voltage 1\", \"units\": \"V\"},\n",
      "        {\"name\": \"Current 0\", \"units\": \"A\"}\n",
      "      ]\n",
      "    },\n",
      "    {\"name\": \"Time\", \"units\": \"msec\", \"values\":[0.0, 0.1, 0.2, 0.3, 0.4, 0.5] },\n",
      "    {\"name\": \"Trial\"},\n",
      "    {\"note\": \"Just some extra info\"}\n",
      "  ]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Accessing Data\n",
      "\n",
      "Data can be accessed through a variety of methods: \\* Standard\n",
      "indexing--You may always just index the array exactly as you would any\n",
      "ndarray \\* Named axes--If you don't remember the order of axes, you may\n",
      "specify the axis to be indexed or sliced like this:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "  data[\"AxisName\":index]\n",
      "  data[\"AxisName\":slice(...)]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`\u00a0Note\u00a0that\u00a0since\u00a0this\u00a0syntax\u00a0hijacks\u00a0the\u00a0original\u00a0slice\u00a0mechanism,\u00a0you\u00a0must\u00a0specify\u00a0a\u00a0slice\u00a0using\u00a0slice()\u00a0if\u00a0you\u00a0want\u00a0to\u00a0use\u00a0named\u00a0axes.`\\\n",
      "`\u00a0*\u00a0Column\u00a0selection--If\u00a0you\u00a0don't\u00a0remember\u00a0the\u00a0index\u00a0of\u00a0a\u00a0column\u00a0you\u00a0wish\u00a0to\u00a0select,\u00a0you\u00a0may\u00a0substitute\u00a0the\u00a0column's\u00a0name\u00a0for\u00a0the\u00a0index\u00a0number.\u00a0Lists\u00a0of\u00a0column\u00a0names\u00a0are\u00a0also\u00a0acceptable.\u00a0For\u00a0example:`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "  data[\"AxisName\":\"ColumnName\"]\n",
      "  data[\"ColumnName\"]  ## Works only if the named column exists for this axis\n",
      "  data[[\"ColumnName1\", \"ColumnName2\"]]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`\u00a0*\u00a0Boolean\u00a0selection--works\u00a0as\u00a0you\u00a0might\u00a0normally\u00a0expect,\u00a0for\u00a0example:`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "  sel = data[\"ColumnName\", 0, 0] > 0.2\n",
      "  data[sel]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`*\u00a0Access\u00a0axis\u00a0values\u00a0using\u00a0!MetaArray.axisValues(),\u00a0or\u00a0.xvals()\u00a0for\u00a0short.\u00a0`\\\n",
      "`*\u00a0Access\u00a0axis\u00a0units\u00a0using\u00a0.axisUnits(),\u00a0column\u00a0units\u00a0using\u00a0.columnUnits()`\\\n",
      "`*\u00a0Access\u00a0any\u00a0other\u00a0parameter\u00a0directly\u00a0through\u00a0the\u00a0info\u00a0list\u00a0with\u00a0.infoCopy()`\n",
      "\n",
      "### File I/O"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "  data.write('fileName')\n",
      "  newData = MetaArray(file='fileName')"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Performance Tips\n",
      "\n",
      "!MetaArray is a subclass of ndarray which overrides the\n",
      "\\`\\_\\_getitem\\_\\_\\` and \\`\\_\\_setitem\\_\\_\\` methods. Since these methods\n",
      "must alter the structure of the meta information for each access, they\n",
      "are quite slow compared to the native methods. As a result, many builtin\n",
      "functions will run very slowly when operating on a !MetaArray. It is\n",
      "recommended, therefore, that you recast your arrays before performing\n",
      "these operations like this:"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "  data = MetaArray(...)\n",
      "  data.mean()                ## Very slow\n",
      "  data.view(ndarray).mean()  ## native speed"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### More Examples\n",
      "\n",
      "`\u00a0\u00a0\u00a0A\u00a02D\u00a0array\u00a0of\u00a0altitude\u00a0values\u00a0for\u00a0a\u00a0topographical\u00a0map\u00a0might\u00a0look\u00a0like`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "      info=[\n",
      "        {'name': 'lat', 'title': 'Latitude'}, \n",
      "        {'name': 'lon', 'title': 'Longitude'}, \n",
      "        {'title': 'Altitude', 'units': 'm'}\n",
      "      ]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`\u00a0\u00a0\u00a0In\u00a0this\u00a0case,\u00a0every\u00a0value\u00a0in\u00a0the\u00a0array\u00a0represents\u00a0the\u00a0altitude\u00a0in\u00a0feet\u00a0at\u00a0the\u00a0lat,\u00a0lon`\\\n",
      "`\u00a0\u00a0\u00a0position\u00a0represented\u00a0by\u00a0the\u00a0array\u00a0index.\u00a0All\u00a0of\u00a0the\u00a0following\u00a0return\u00a0the\u00a0`\\\n",
      "`\u00a0\u00a0\u00a0value\u00a0at\u00a0lat=10,\u00a0lon=5:`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "      array[10, 5]\n",
      "      array['lon':5, 'lat':10]\n",
      "      array['lat':10][5]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`\u00a0\u00a0\u00a0Now\u00a0suppose\u00a0we\u00a0want\u00a0to\u00a0combine\u00a0this\u00a0data\u00a0with\u00a0another\u00a0array\u00a0of\u00a0equal\u00a0dimensions\u00a0that`\\\n",
      "`\u00a0\u00a0\u00a0represents\u00a0the\u00a0average\u00a0rainfall\u00a0for\u00a0each\u00a0location.\u00a0We\u00a0could\u00a0easily\u00a0store\u00a0these\u00a0as\u00a0two\u00a0`\\\n",
      "`\u00a0\u00a0\u00a0separate\u00a0arrays\u00a0or\u00a0combine\u00a0them\u00a0into\u00a0a\u00a03D\u00a0array\u00a0with\u00a0this\u00a0description:`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "      info=[\n",
      "        {'name': 'vals', 'cols': [\n",
      "          {'name': 'altitude', 'units': 'm'}, \n",
      "          {'name': 'rainfall', 'units': 'cm/year'}\n",
      "        ]},\n",
      "        {'name': 'lat', 'title': 'Latitude'}, \n",
      "        {'name': 'lon', 'title': 'Longitude'}\n",
      "      ]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "`\u00a0\u00a0\u00a0We\u00a0can\u00a0now\u00a0access\u00a0the\u00a0altitude\u00a0values\u00a0with\u00a0array[0]\u00a0or\u00a0array['altitude'],\u00a0and\u00a0the`\\\n",
      "`\u00a0\u00a0\u00a0rainfall\u00a0values\u00a0with\u00a0array[1]\u00a0or\u00a0array['rainfall'].\u00a0All\u00a0of\u00a0the\u00a0following\u00a0return`\\\n",
      "`\u00a0\u00a0\u00a0the\u00a0rainfall\u00a0value\u00a0at\u00a0lat=10,\u00a0lon=5:`"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "      array[1, 10, 5]\n",
      "      array['lon':5, 'lat':10, 'val': 'rainfall']\n",
      "      array['rainfall', 'lon':5, 'lat':10]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "    Notice that in the second example, there is no need for an extra (4th) axis description\n",
      "    since the actual values are described (name and units) in the column info for the first axis.\n",
      "\n",
      "=== Contact ===\n",
      "Luke Campagnola - lcampagn@email.unc.edu \n"
     ]
    }
   ],
   "metadata": {}
  }
 ]
}